@hpcc-js/wasm
Advanced tools
This repository contains a collection of useful c++ libraries compiled to WASM for (re)use in Node JS, Web Browsers and JavaScript Libraries:
Built with:
The simplest way to include this project is via NPM:
npm install --save @hpcc-js/wasm
The @hpcc-js/wasm package includes the following files in its dist folder:
index.js / index.min.js: Browser UMD Package for all APIs.index.es6.js: Browser ESM Package for all APIs.index.node.js: Node CJS Package for all APIs.index.node.es6.js: Node ESM Package for all APIs.graphviz.js: Browser UMD Package for graphviz APIs.graphviz.es6.js: Browser ESM Package for graphviz APIs.graphvizlib.wasm: graphviz wasm file (loaded on demand).expat.js: Browser UMD Package for expat API.expat.es6.js: Browser ESM Package for expat API.expatlib.wasm: expat wasm file (loaded on demand).expat.js: Browser UMD Package for zstd API.zstd.es6.js: Browser ESM Package for zstd API.zstdlib.wasm: zstd wasm file (loaded on demand).Important: WASM files are dynamically loaded at runtime (this is a browser / emscripten requirement), which has a few implications for the consumer:
Pros:
Cons:
fetch and loading pages via file:// URN, so for testing / development work you will need to run a test web server.Alternatively the @hpcc-js/wasm package can be imported directly within the html page, using a NPM CDN server like unpkg, jsdelivr. For modern browsers and import:
<script type="module">
import { graphvizSync } from "https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/index.es6.js";
graphvizSync().then(graphviz => {
const div = document.getElementById("placeholder2");
div.innerHTML = graphviz.layout(dot, "svg", "dot");
});
</script>
For legacy environments you can load the UMD packages:
<script src="https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/index.min.js"></script>
<script>
var hpccWasm = window["@hpcc-js/wasm"];
hpccWasm.graphvizSync().then(graphviz => {
var div = document.getElementById("placeholder2");
div.innerHTML = graphviz.layout(dot, "svg", "dot");
});
</script>
GraphViz WASM library, see graphviz.org for c++ details. While this package is similar to Viz.js, it employs a completely different build methodology derived from GraphControl.
To call dot-wasm without installing:
npx -p @hpcc-js/wasm dot-wasm [options] fileOrDot
To install the global command dot-wasm via NPM:
npm install --global @hpcc-js/wasm
Usage:
Usage: dot-wasm [options] fileOrDot
Options:
--version Show version number [boolean]
-K, --layout Set layout engine (circo | dot | fdp | sfdp | neato | osage | patchwo
rk | twopi). By default, dot is used.
-T, --format Set output language to one of the supported formats (svg | dot | json
| dot_json | xdot_json | plain | plain-ext). By default, svg is prod
uced.
-n, --neato-no-op Sets no-op flag in neato.
"-n 1" assumes neato nodes have already bee
n positioned and all nodes have a pos attribute giving the positions.
It then performs an optional adjustment to remove node-node overlap,
depending on the value of the overlap attribute, computes the edge l
ayouts, depending on the value of the splines attribute, and emits th
e graph in the appropriate format.
"-n 2" Use node positions as speci
fied, with no adjustment to remove node-node overlaps, and use any ed
ge layouts already specified by the pos attribute. neato computes an
edge layout for any edge that does not have a pos attribute. As usual
, edge layout is guided by the splines attribute.
-y, --invert-y By default, the coordinate system used in generic output formats, suc
h as attributed dot, extended dot, plain and plain-ext, is the standa
rd cartesian system with the origin in the lower left corner, and wit
h increasing y coordinates as points move from bottom to top. If the
-y flag is used, the coordinate system is inverted, so that increasin
g values of y correspond to movement from top to bottom.
-v Echo GraphViz library version
-h, --help Show help [boolean]
Examples:
dot-wasm -K neato -T xdot ./input.dot Execute NEATO layout and outputs XDOT format.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>GraphViz WASM</title>
<script src="https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/index.min.js"></script>
<script>
var hpccWasm = window["@hpcc-js/wasm"];
</script>
</head>
<body>
<div id="placeholder0"></div>
<script>
const test = `\
digraph {
layout = neato
splines = true
edge [len = 2]
a -> b
b -> a
}`;
hpccWasm.graphviz.layout(test, "svg", "dot").then(svg => {
const div = document.getElementById("placeholder0");
div.innerHTML = svg;
});
</script>
<div id="placeholder1"></div>
<div id="placeholder2"></div>
<script>
const dot = `
digraph G {
node [shape=rect];
subgraph cluster_0 {
style=filled;
color=lightgrey;
node [style=filled,color=white];
a0 -> a1 -> a2 -> a3;
label = "Hello";
}
subgraph cluster_1 {
node [style=filled];
b0 -> b1 -> b2 -> b3;
label = "World";
color=blue
}
start -> a0;
start -> b0;
a1 -> b3;
b2 -> a3;
a3 -> a0;
a3 -> end;
b3 -> end;
start [shape=Mdiamond];
end [shape=Msquare];
}
`;
// Asynchronous call to layout
hpccWasm.graphviz.layout(dot, "svg", "dot").then(svg => {
const div = document.getElementById("placeholder1");
div.innerHTML = svg;
});
</script>
<script type="module">
import { graphvizSync } from "https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/index.es6.js";
graphvizSync().then(graphviz => {
const div = document.getElementById("placeholder2");
// Synchronous call to layout
div.innerHTML = graphviz.layout(dot, "svg", "dot");
});
</script>
</body>
</html>
The GraphViz library comes in two flavours
graphviz namespace, where each API function is asynchrounous and returns a Promise<string>.graphvizSync asynchrounous function which returns a Promise<GraphvizSync> which is a mirror instance of graphviz, where each API function is synchrounous and returns a string.Returns the Graphviz Version.
# layout(dotSource[, outputFormat][, layoutEngine][, ext]) · <>
Performs layout for the supplied dotSource, see The DOT Language for specification.
outputFormat supports the following options:
See Output Formats for more information.
layoutEngine supports the following options:
See Layout manual pages for more information.
ext optional "extra params":
array of{
path: string; // The path for the image.
width: string; // Width of Image
height: string; // Height of Image
}
array of{
path: string; // The path for the file.
data: string; // The data for the file.
}
string specifying the location of wasm file.XHR or fetch.For example passing a web hosted Image to GraphViz:
hpccWasm.graphviz.layout('digraph { a[image="https://.../image.png"]; }', "svg", "dot", {
images: [{
path: "https://.../image.png",
width: "272px",
height: "92px"
}]
}).then(svg => {
document.getElementById("placeholder").innerHTML = svg;
}).catch(err => console.error(err.message));
# circo(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs circo layout, is equivalent to layout(dotSource, outputFormat, "circo");.
# dot(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs dot layout, is equivalent to layout(dotSource, outputFormat, "dot");.
# fdp(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs fdp layout, is equivalent to layout(dotSource, outputFormat, "fdp");.
# sfdp(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs sfdp layout, is equivalent to layout(dotSource, outputFormat, "sfdp");.
# neato(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs neato layout, is equivalent to layout(dotSource, outputFormat, "neato");.
# osage(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs osage layout, is equivalent to layout(dotSource, outputFormat, "osage");.
# patchwork(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs patchwork layout, is equivalent to layout(dotSource, outputFormat, "patchwork");.
# twopi(dotSource[, outputFormat][, ext]) · <>
Convenience function that performs twopi layout, is equivalent to layout(dotSource, outputFormat, "twopi");.
# graphvizSync([wasmFolder], [wasmBinary]) · <>
Returns a Promise<GraphvizSync>, once resolved provides a synchronous variant of the above methods. Has an optional wasmFolder argument to override the default wasmFolder location and optional wasmBinary to short circuit the wasm downloading process.
Expat WASM library, provides a simplified wrapper around the Expat XML Parser library, see libexpat.github.io for c++ details.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>GraphViz WASM</title>
<script src="https://unpkg.com/@hpcc-js/wasm/dist/index.min.js"></script>
<script>
var hpccWasm = window["@hpcc-js/wasm"];
</script>
</head>
<body>
<script>
const xml = `
<root>
<child xxx="yyy">content</child>
</root>
`;
var callback = {
startElement(tag, attrs) { console.log("start", tag, attrs); },
endElement(tag) { console.log("end", tag); },
characterData(content) { console.log("characterData", content); }
};
hpccWasm.parse(xml, callback);
</script>
</body>
</html>
Returns the Expat Version.
Parses the XML with suitable callbacks.
Note: characterData may get called several times for a single tag element.
zstd for short
Zstandard WASM library, provides a simplified wrapper around the Zstandard c++ library, see Zstandard for more details.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Zstandard WASM</title>
</head>
<body>
<div id="placeholder"></div>
<script type="module">
import { Zstd } from "https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/index.es6.js";
const zstd = await Zstd.load();
const data = new Uint8Array(Array.from({ length: 100000 }, (_, i) => i % 256));
const compressed_data = await zstd.compress(data);
const decompressed_data = await zstd.decompress(compressed_data);
document.getElementById("placeholder").innerHTML = `\
<ul>
<li>Default Compression Level: ${await zstd.defaultCLevel()}</li>
<li>Decompressed Size (bytes): ${decompressed_data.byteLength}</li>
<li>Data Size (bytes): ${data.byteLength}</li>
<li>Compressed Size (bytes): ${compressed_data.byteLength}</li>
<li>Decompressed Size (bytes): ${decompressed_data.byteLength}</li>
</ul>
`;
</script>
</body>
</html>
Options structure for advanced loading.
interface Options {
wasmFolder?: string;
wasmBinary?: ArrayBuffer;
}
string specifying the location of wasm file.XHR or fetch.Conceptual interface for TypeScript/JavaScript wrapper API
interface Zstd {
static load(options?: Options): Promise<Zstd>;
version(): string;
compress(data: Uint8Array, compressionLevel: number = this.defaultCLevel()): Uint8Array;
decompress(array: Uint8Array): Uint8Array;
defaultCLevel(): number;
}
# Zstd.load(options?: Options): Promise<Zstd> · <>
Loads and initializes the Zstandard wasm library, returns a Promise to Zstd:
const zstd = await Zstd.load();
...dostuff...
or
Zstd.load().then(zstd => {...dostuff...});
# zstd.compress(data: Uint8Array, compressionLevel?: number): Uint8Array · <>
zstd.defaultCLevel() which is currently 3.Compresses raw data.
A note on compressionLevel: The library supports regular compression levels from 1 up 22. Levels >= 20, should be used with caution, as they require more memory. The library also offers negative compression levels, which extend the range of speed vs. ratio preferences. The lower the level, the faster the speed (at the cost of compression).
# zstd.defaultCLevel(): number · <>
Similar to Base 64, but uses more characters resulting in smaller strings.
Base 91 WASM library, similar to Base 64 but uses more characters resulting in smaller strings, see Base91 for more details.
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Base91 WASM</title>
</head>
<body>
<div id="placeholder"></div>
<script type="module">
import { Base91 } from "./dist/index.es6.js";
//import { Base91 } from "https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/index.es6.js";
const base91 = await Base91.load();
const data = new Uint8Array(Array.from({ length: 100 }, (_, i) => Math.random() * 100));
const encoded_data = await base91.encode(data);
const decoded_data = await base91.decode(encoded_data);
document.getElementById("placeholder").innerHTML = `\
<ul>
<li>Data Size (bytes): ${data.byteLength}</li>
<li>Endoded Size (bytes): ${encoded_data.length}</li>
<li>Decoded Size (bytes): ${decoded_data.byteLength}</li>
</ul>
<h4>Data: </h4>
<code>
${data}
</code>
<h4>Base 91: </h4>
<code id="base91">
</code>
<h4>Decoded: </h4>
<code>
${decoded_data}
</code>
`;
document.getElementById("base91").innerText = encoded_data;
</script>
</body>
</html>
Options structure for advanced loading.
interface Options {
wasmFolder?: string;
wasmBinary?: ArrayBuffer;
}
string specifying the location of wasm file.XHR or fetch.Conceptual interface for TypeScript/JavaScript wrapper API
interface Base91 {
static load(options?: Options): Promise<Base91>;
version(): string;
encode(data: Uint8Array): string;
decode(array: string): Uint8Array;
}
# Base91.load(options?: Options): Promise<Base91> · <>
Loads and initializes the Base91 wasm library, returns a Promise to Base91:
const base91 = await Base91.load();
...dostuff...
or
Base91.load().then(base91 => {...dostuff...});
# base91.version(): string · <>
# base91.encode(data: Uint8Array): string · <>
Encodes the raw data.
# base91.decode(str: string): Uint8Array · <>
Decodes the raw data.
Utility functions unrelated to any specific wasm APIs
If url is specified, sets the default location for all WASM files. If url is not specified it returns the current url (defaults to undefined).
Global variable for setting default WASM location, this is an alternative to wasmFolder
Building is supported on both Linux (tested with Ubuntu 20.04) and Windows with WSL enabled (Ubuntu-20.04). Building in other environments should work, but may be missing certain prerequisites.
These are then known required OS dependencies (see ./docker/ubuntu-dev.dockerfile for test script):
sudo apt-get install -y curl
sudo curl --silent --location https://deb.nodesource.com/setup_16.x | sudo bash -
sudo apt-get install -y nodejs
sudo apt-get install -y build-essential
sudo apt-get install -y git cmake wget
sudo apt-get install -y gcc-multilib g++-multilib pkg-config autoconf bison libtool flex zlib1g-dev
sudo apt-get install -y python3 python3-pip
git clone https://github.com/hpcc-systems/hpcc-js-wasm.git
cd hpcc-js-wasm
npm ci
npm run install-build-deps
npm run build
Note: The install-build-deps downloads the following dependencies:
This has been made a manual step as the downloads are quite large and the auto-configuration can be time consuming.
It is worth noting that npm run clean will only clean any artifacts associated with the build, but won't clean clean any of the third party dependencies. To remove those for a "full clean", run:
npm run uninstall-build-deps
FAQs
hpcc-js - WASM Libraries
The npm package @hpcc-js/wasm receives a total of 42,278 weekly downloads. As such, @hpcc-js/wasm popularity was classified as popular.
We found that @hpcc-js/wasm demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Ransomware payment rates hit an all-time low in 2024 as law enforcement crackdowns, stronger defenses, and shifting policies make attacks riskier and less profitable.
Security News
require(esm) backported to Node.js 20, easing the transition to ESM-only packages and reducing complexity for developers as Node 18 nears end-of-life.
Security News
PyPI now supports iOS and Android wheels, making it easier for Python developers to distribute mobile packages.